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Abstract. We examine a previouly introduced attractor neural network model that 
explains the persistent activities of neurons in the anterior ventral temporal cortex 
of the brain. In this model, the coexistence of several attractors including correlated 
attractors was reported in the cases of finite and infinite loading. In this paper, by 
means of a statistical mechanical method, we study the statics and dynamics of the 
model in both finite and extensive loading, mainly focusing on the retrieval properties 
of the Hopfield and correlated attractors. In the extensive loading case, we derive the 
evolution equations by the dynamical replica theory. We found several characteristic 
temporal behaviours, both in the finite and extensive loading cases. The theoretical 
results were confirmed by numerical simulations. 



PACS numbers: 87.10, 05.20, 84.35 

E-mail: uezu@ki-rin . phys . nara-wu .ac.jp 

1. Introducion 

The attractor neural network model has been considered a plausible model for memory 
processing in the brain. Many experimental findings implying its existence have been 
reported. Among them, Miyashita and his group reported the persistent activities of 
neurons in the anterior ventral temporal ( AVT) cortex. P3 |2] They trained monkeys to 
recognize and match a set of 100 colored fractal patterns in the Delayed Match to Sample 
(DMS) paradigm. Miyashita and Chang reported that the neurons in the AVT are highly 
selective toward a few of the 100 colored fractal patterns, and that the activities of these 
neurons persist for 16 seconds after the removal of the presented stimulus. [2] Miyashita 
studied development of this selectivity, and found that serial positions number of stimuli 
during learning are converted into the spatial correlation of the neural activities. jTj 

To explain these experiments, Griniasty et al. proposed a model for an 
attractor neural network. [3] By analysing equilibrium states of this model in the 
finite loading case, they found not only stored patterns (Hopfield attractors) but 
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also patterns which have overlap with stored patterns (correlated attractors). For 
example, when the number of patterns is 13, correlated attractors are expressed by 
m = 2f*(77, 51, 13, 3, 1, 0, 0, 0, 0, 1, 3, 13, 51) and its cyclic rotations, whereas Hopfield 
attractors are expressed by m = l (1, 0, • • • , 0) and its cyclic rotations. Here, m is the 
overlap vector m =* (mi,m2, ■ • ■ ,m c ), of which component m M is the overlap between 
the state of neurons and the /x-th pattern, t denotes transposition. See eq. (JHJ). Griniasty 
et al. found that the correlation between correlated attractors dicreases as the separation 
in the sequence of the patterns to which the attractors belong increases. These results 
support the work done by Miyashita. 

In recent years, we have studied the statics and dynamics of the model introduced by 
Griniasty et al. in the finite loading case. As for the statics, we investigated equilibrium 
states and found the coexistence of several attractors such as Hopfield attractors, 
correlated attractors, and mixed states of several patterns in several parameter regions. 
The coexistence of attractors was also reported by Shiino et al. |U |3] and by Fukai 
et al.[6j As for the dynamics, we investigated the transient behaviour by performing 
numerical calculations of the overlap dynamics derived theoretically and by numerical 
simulations. In particular, we studied what happens when a Hopfield attractor and a 
correlated attractor coexist and when a Hopfield attractor does not exist and a correlated 
attractor does exist. In the former case, we studied the basin of attraction and in the 
latter case, we found that the trajectory initially approaches the state where the Hopfield 
attractor existed, but finally it tends toward the correlated attractor. We found that the 
results by numerical calculations of the overlap dynamics and the results by numerical 
simulations agree qualitatively. 

The coexistence of multiple attractors and the dynamical behaviour obtained in 
the finite loading case are significant. However, the situation of the finite loading seems 
to be unrealistic. Thus, it is necessary to investigate whether the results in the finite 
loading case are obtained when applied practically, i.e. in the extensive loading case. 

In this paper, we study the model introduced by Griniasty et al. by the statistical 
mechanical method, mainly focusing on the extensive loading case. In the extensive 
loading; to the study of the equilibrium states, Cugliandolo et al. derived 

the saddle point equations(S.P.E.) for order parameters using the replica method. [Zj 
They identified not only the Hopfield attractors but also the correlated attractors. We 
have solved the saddle point equations numerically by scanning parameters and found 
that there are parameter regions where several attractors coexist. The coexistence of 
multiple attractors was found by Shiino et al. in the present model as well. |3J [3] As for 
the dynamical behaviour, we have derived evolution equations for order parameters by 
using the dynamical replica theory by Laughton and Coolen. [H] As in the finite loading 
case, we have studied the dynamical behaviour in a coexistence region of a Hopfield 
attractor and a correlated attractor, and also in a region where a Hopfield attractor does 
not exist but a correlated attractor does exist. From these studies, we have obtained 
qualitatively similar results to those in the finite loading case. 

In the next section, we explain the present model. Then, we summarize the theory 
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and results for the finite loading case in §3. In §4, the extensive loading case is examined. 
The summary and discussion are given in §5. 

2. Model 



Let us explain the model. The instantaneous state of each neuron is expressed by 
which takes ±1, where i labels the neuron(i = 1, • • • , iV), and the time evolution is given 
by 

Si (f + 1) =sign(^(*)), (1) 

where 

hi(t) = E -W*), (2) 

and Jij is the strength of the synaptic connection from the j-th neuron to the i-th neuron. 
The stochastic dynamics is also taken into consideration by introducing temperature T, 
that is, the probability that Si(t + 1) takes ±1 is given by 

1 ±tanh(/3/i;(t)) 



Probst + 1)] 



(3) 



where (5 — 1/T. In the present model, the synaptic weight is defined as 

where £f represents the value of i-th neuron for /z-th pattern ^ = , • • • , and it 
takes value +1 or -1 with a probability of 1/2. c is the total number of patterns. A is 
a c x c matrix defined as 

/ 1 a • • • a \ 
a 1 a • • • 
a 1 a 
a 1 a 



{ V) = 



V a 



(5) 



a 1 J 

(m 1 ,m 2 , 



The order parameter is the overlap vector m 

is the overlap between the state of neurons s =* (s 1; • 
that is, 

1 w 

1=1 



• , m c ), of which component 
■ , Sjv) and the /x-th pattern 



(6) 



3. Finite Loading Case 

In this section, we study the finite loading case of the model described by eqs.(l)-(4). 
That is, we consider what happens when N ^ 1 and a = | < 1. 
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3.1. Theory 

To study the equilibrium state, we consider the following Hamiltonian H. 

H = ~ j E J H s i s i ( 7 ) 

= -^EEefv^- (8) 

ZJV ¥i w 

We calculate the free energy of the system 

/ = --L]nZ, Z = Tr s e-^, 

where Tr s denotes the summation with respect to Si(i = 1, ■ ■ • , N). By using the saddle 

point method, we obtain [9 

ft c c 

/ = - m^A^rriy - i ^ m^m^ - [ln{2 cosh(-i Yl "Vf ( 9 ) 

where £ =* £ 2 , • • • , £ c ) and [A]^ is the average over £, that is [A]g = i S{£m=±i}A. 
Then, the S.P.E. becomes 

c 

m^ = ipJ2 A m>™vi ( 10 ) 
i/=i 



[^tanh(/3 £ rA^rrv)]^ (U) 
i/,i/=i 

The physical meaning of m M is the overlap between the equilibrium state and the /i -th 
pattern, 

1 N 

v£tf<*>. (12) 

where (sj) is the thermal average of the i-ih neuron. 

Next, we derive the evolution equation of the system. Let us consider the probability 
Vt{m) that at time t the state s has the overlap m M with /i-th pattern £ M for // = 1, • • • , c 



P t (m) = TWO J] 5(m^ - m»), (13) 
1 N 

™,( s ) = ^Etf*, ( 14 ) 

iV i=l 

where Pt(s) is the probability that the system takes the state s at time t. We assume 

that the transition probability u>j(s) from the state s = (si, • • • , Sj, • • • , Sjv) to the state 
FjS = (s]_, • ■ ■ , —Si, • • - , s^) takes the following form 

1 - Sjtanh{/?/ii(s)} 

W) = ^ , (15) 

where Fj is the flip operator of the i-th neuron and hi(s) is expressed as 

TV 

k i( S ) = £ J i3 S i ~ £ 6fA^ m *( S )" ( 16 ) 
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The master equation for pt(s) is given by 
d N 

j t Pt(s) = ^{w^sMFis) - Wi (s)p t (s)}. (17) 

Then, using asynchronous dynamics, we obtain the evolution equations for the overlap 
m as 

^m= -m+ [£tanh(/?'£4m)]£. (18) 

The equation for the stationary state of eq. (fTHj) agrees to the eq. (fTTj) for the equilibrium 

state. 

3.2. Results 

c is finite and we put c = 13 as in ref . [7] , becasuse the correlated attractors obtained by 
Amit et al. have non-zero values up to their fifth nearest neighbours. 

The equilibrium state 

We solved the S.P.E. flllj) numerically and found the coexistence of several attractors 
such as the Hopfield attractors, correlated attractors, and mixed states of several 
patterns in several parameter regions. For example, we show the result for a = 0.4 
in fig. 1. The solid curve denotes the Hopfield attractor. It exists up to T ~ 0.1. The 
dashed curves denote a correlated attractor. For this attractor, there is the following 
symmetry 

m 2 = m 13 , m 3 = m 12 , ■ ■ ■ ,m 7 = m 8 . 

This attractor exists up to T ~ 0.25. It has been considered that correlated attractors 
exist only for a > 0.5. [7] However, we found that they exist even for a < 0.5. As is 
seen from the figure, up to T ~ 0.1, both the Hopfield attractor and the correlated 
attractor exist. Further, we found that the mixed states of three patterns and 
£ M+2 (which are not drawn in fig. 1) exist up to T ~ 0.05 and the mixed state of 13 
patterns exists up to T ~ 1.7. 

Dynamics 

First, we investigate the case where several attractors coexist. For example, for 
a = 0.4 and T = 0.04, a Hopfield attractor, a correlated attractor, mixed states of three 
patterns and a mixed state of 13 patterns coexist. For this parameter, we investigate 
the basin of attraction of these attractors by using overlap dynamics (eq. (fTHjl ) and 
numerical simulations. In the numerical integrations of eq. (|T8|) . we set the initial 
condition as 

m^t = 0) = m ^,i, )U = 1, (19) 

On the other hand, in the numerical simulations, we generate an initial state s(0) 
according to the following probability, 

Probfo = ±1] = 1± ™°£ . (20) 
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When iV — > oo, the overlap m M between s(0) and £ M satisfies the relation (|19|). 
In fig. 2, we show the results by numerical integrations and those by numerical 
simulations. In fig. 3, we compare the results by numerical integrations and those 
by numerical simulations in more detail. As is shown in fig. 3(a), in the numerical 
integrations, when the initial overlap with the pattern 1, mo, is 0.15, the trajectory 
tends toward the correlated attractor and the trajectory with m = 0.16 tends toward 
the Hopfield attractor. That is, the boundary between the basin of attraction for the 
Hopfield attractor and that for the correlated attractor, m^, is between 0.15 and 0.16. 
We show the results of the simulations in fig. 3(b). When m is 0.16, the trajectory 
tends toward the correlated attractor and when mo is 0.17, the trajectory tends toward 
the Hopfield attractor. From these results, we note that even if the trajectories finally 
tend toward the correlated attractor, they initially approach the Hopfield attractor. 

Next, we investigate the case where a Hopfield attractor does not exist but a 
correlated attractor exists, which is shown in fig. 4. As the figures show, both in the 
numerical integrations of eq. (fT%|) and numerical simulations, we found that for mo < 1, 
the trajectory initially approaches the state where the Hopfield attractor existed, but 
finally it tends toward the correlated attractor. 

In the next section, we study the extensive loading case to see whether the 
characteristic feature in the finite loading case such as the coexistence of multiple 
attractors and characteristic temporal behaviour continues to exist or not. 



4. Extensive Loading Case 

4-1. Theroy 

As an example of an extensive loading case, we consider the following synaptic weight 

Jijj 

c is the number of condensed patterns and we set c = 13. p is the total number of 
patterns and we consider the case that a = jr is finite. We assume that £f and r/f take 
+ 1 or -1 with a probability of 1/2. The overlap m A1 (s) between the state of neurons s 
and the /x-th pattern £ M is defined as 

1 N 

= — J^Siff for /i = l,---,c 
1 N 

= Jr22 s Mi for /i = c+l,---,p. 

i=l 

The cross-talk noise Zi(s) is defined as 

z i( s ) = Y. ll> jii m ^ s )- 
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As order parameters, we define m =' (m 1; m 2 , ■ ■ • , m c ) and r = - Y^= c +i( m fj.( s )) 2 where 
ar is the variance of the cross-talk noise. 



The equilibrium states 

By using the replica method, the free energy / for the replica symmetric(RS) 
solution [7j is given as 



-T[J Dz\n{2 cosh f3(^/blr~z + l (Am)}]^ (21) 

where £ =* (C 1 , • • • , C°) an d is the average over £, that is ^T,^ = ±iyA. The S.P.E. 
for m, g and r are given by 



m = [C f D z t&nh (3 Am + y/afz)]^, (22) 
q = [J Dzttmh 2 (3{ t C,Am + y/arz)]^, (23) 

- ( 24 ) 



T-c 



where c = (3(1 — q), Dz = dze x I 2 ' j\p2/K and A is the c x c matrix which is defined in 
eq. (JHJ). For T = 0, these equations become 



^-[exp{-(^) 2 }] C , (25) 



m 

'or 



;i - cf 



where = £° fa-*". 



(27) 



Dynamics 

As for the dynamics, we derive the evolution equations for (m, r) using the 
dynamical replica theory(DRT).|B a We assume that the same transition probability u>;(s) 
as in the finite loading case, but in the present case, hi(s) is expressed as 

hi(s) = £ gA^m^J) + Zi(a). (28) 

The master equation for pt(s) is the same as before. Using asynchronous dynamics, we 
obtain the evolution equations for the overlap m and the variance of the cross-talk noise 
ctr(s), assuming the self-averaging in the limit of iV — > oo. Then, we obtain 

d r 

— m= -m + J dz[D^[z]C tanhpCC Am + z)]^, (29) 



d 
dt 



-r + l + ^-J dz[V^[z]z tanh (3{ l C Am + z)]^, (30) 
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where is the probability density function of the cross-talk noise Zi(s) 

\ i V m,r;t 

dz[D c [z)) c = 1, 

where £ 4 =* • • • , £?). ($)m,r;t denotes the subshell average 

Es p t {s)S(m - m(s))5{r - r(s))$(s) 



J2sPt(s)S(m-m(s))5(r-r(s)) 
Here, to obtain closed equations, we assume the following: 

(i) The time evolutions of the order parameters (m, r) have the self-averaging property 
with respect to the average over the variables £f . 

(ii) The probability distribution of the micro variable s is independent of time and is 
uniform in the subshell where the macro variables (m, r) have the same values. 

Then, we obtain the closed equations for (m, r) using the replica method under the RS 
ansatz, 



|m = - m + [</ D X J D y{l- tanhCCM + + ^*)} 

x tanh/3(*CAra + [/-)]£, (31) 



= - r + 1 + [/ Dx J Dy l -{1 - tanh^M + yJ±X + ^*)} 

x ET tanh/3(*CAm + C/~)]^, (32) 

where e = A = , U~(x) = \[2ex — A. When (m, r) are given, the parameters 

A, p, q and /x are determined using the following equations. 



A " i-p(i-g)' (33) 



l-p(l-g) 



2 



' [i-p(i-*)F (34) 

g = [J Dwt&nh 2 ( t Cv + Aw)]^, (35) 

m=[f £>u<tanh(*CM + Aw)]^. (36) 

From eq. (jSJ), p is given by 

P = 2r{ l_ q) [2r - 1 + g - v/(l-g) 2 + 4rg] > 0. (37) 

In particular, the evolution equations for T = is given by 

= -m+[C [ DyiT -^Le-^ u+x - )2 (l-t&nh(E(u + x„,y,t))) 

dt J JO yln 

<IU -_e-W-*-? (1 - tanh(~(-( M - x_), y, 0))}U 
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1 d 
2dt 



-r + l + 

f°° du 



-[ / Dy{ / —. Le-^ u+x ^ (1 - tanh(H(u + x_, y, 0)) ET (u + a?_) 
a J jo v27r 



r v_i (u _ x _ f _ tanh(H( _ M + c))) v - { _ u + x _ )}] 

Jo v2tt "» 



27T 

A 2 



(39) 



where E(x,y,C) = C^ + VyJ^ + and x_ = -^=(*CAm- A). 

In the next subsection, we give the results for T = by the numerical calculations 
of the S.P.E., and also the results of the evolution equations together with the results 
by numerical simultions. 



Jf.,2. Results 

The equilibrium states 

We solved the S.P.E. (j25|) - (|2*7j) numerically for several values of parameters a and a 
and found the coexistence region of correlated attractors, Hopfield attractors, and mixed 
states. In fig. 5, we show an example of the phase diagrams. When a is decreased, at 
a ~ 0.0183, a pair of correlated attractors appear. One of the correlated attractors con- 
tinues to exist at least until a ~ 0.0014. This attractor is not shown in fig. 5 because it 
is unstable, that is, this is not an attractor but a repeller. Another correlated attractor, 
which is drawn in the figure, is stable and exists until a ~ 0.0049. At a ~ 0.0049, 
this correlated attractor disappears and then a mixed state with three patterns appears 
and continues to exist until a = 0. Further, a mixed state with thirteen patterns exist 
for a G (0,0.3119). The Hopfield attractor exists only for a G (0,0.013). Thus, for 
a G (0.0049,0.013), the Hopfield attractor, a correlated attractor, and a mixed state 
with thirteen patterns coexist. 



Dynamics 

We studied the following two cases using deterministic dynamics. 
Case 1 A Hopfield attractor and a correlated attractor coexist. 
Case 2 A correlated attractor, but no Hopfield attractor. 

In both cases, we performed numerical integrations of the evolution equations (}3*8*j) 
and ()39)1 and numerical simulations of eq. ([TJ. In the numerical integration, we took 
m(0) = (m , 0, • • • , 0) and r(0) = 1 as an initial condition. Since the integration of the 
evolution equations required a lot of computation time, we used the Euler method as 
the integration scheme. 

In the numerical simulations, an initial value of s« was determined according to the 
probability distribution ()2()jl . In both numerical integrations and numerical simulations, 
m was taken as m = 0.1, 0.2, • • • , 0.9. 

Case 1 

As an example, we set a = 0.35, a = 0.01. In fig. 6, we show the result of DRT 
and of numerical simulations. As the figure shows, the boundary between the basin 
of attraction for the Hopfiled attractor and that for the correlated attractor is around 
mi(0) = 0.5. Except for the boundary of the basin of attraction, the results by two 
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methods agree fairly well. 
Case 2 

As an example, we set a = 0.35 and a = 0.015. As is seen from fig. 7, the 
trajectories initially approach the state where the Hopfield attractor existed but finally 
tend toward the correlated attractor in both the numerical integrations and numerical 
simulations except for m = 0.9 in the numerical integration. In the case of m = 0.9 
in the numerical integration, our numerical scheme to solve the saddle point equations 
did not generate a solution at some time. We suggest that this is due to the precision 
of the numerical calculations. 

The tendency was that trajectories obtained by numerical integrations tend toward the 
correlated attractor faster than those obtained by numerical simulations. 

5. Summary and Discussion 

We studied the statics and dynamics of the model introduced by Griniasty et al. in 
the finite and extensive loading cases. In both cases, we qualitatively obtained similar 
results. First, we summarize the results common to both cases. 

As for the statics, we solved the saddle point equations numerically and obtained 
equilibrium states. We found regions of parameters where the Hopfield attractor and a 
correlated attractor coexist. It had been previouly believed that correlated attractors 
exist only for a > 0.5. However, we found that the correlated attractors exist not only 
for a > 0.5 but also for a < 0.5. 

As for the dynamics, we studied the following two cases by numerical integrations 
of the evolution equations and by the numerical simulations, deriving the evolution 
equations by dynamical replica theory in the extensive loading case. 

First, we studied the transient behaviour when a Hopfield attractor and a correlated 
attractor coexist. We found that when we set initial conditions at m(0) = (m , 0, • • • , 0) 
( and r(0) = 1 for the extensive loading case), there exists some critical initial overlap 
uIq, which is the boundary between the basin of attraction for the Hopfield attractor 
and that for the correlated attractor. Next, we studied the transient behaviour when a 
Hopfield attractor does not exist but a correlated attractor does exist. Then, we found 
that trajectories initially approach the state where the Hopfield attractor existed but 
finally they tend toward the correlated attractor. 

Next, we compare the results obtained in the finite loading case and the results 
obtained in the extensive loading case. 
Equilibrium state 

In both cases, the Hopfield attractor, the mixed state, and the correlated attractor 
exist, and there are parameter regions where the Hopfield attractor, the mixed state, 
and the correlated attractors coexist. 
Dynamics 

1. When several attractors coexist 

In the finite loading case (a = 0.4, T = 0.04), the boundary between the 
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basin of attraction for the Hopfiled attractor and that for the correlated attractor is 
0.15 < ml < 0.16 in the numerical integrations and 0.16 < ml < 0.17 in the numerical 
simulations. On the other hand, in the extensive loading case(a = 0.35, a = 0.01, T = 0), 
the boundary between the basin of attraction for the Hopfiled attractor and that for 
the correlated attractor is 0.4 < ml < 0.5 in both the numerical integrations and 
the numerical simulations. In the finite and extensive loading cases, the results of 
numerical integrations and the numerical simulations agree fairly well. One reason 
for the difference between the finite and extensive loading cases might be due to the 
difference in parameters. 

2. When a Hopfield attractor does not exist but a correlated attractor does 
exist 

In the finite loading case, we used mi(0) = 0.1, 0.2, • • • , 1.0 as the initial conditions. 
Then the trajectories finally tended toward the correlated attractor in both the numeri- 
cal integrations and numerical simulations. On the other hand, for the extensive loading 
case (a = 0.35, a = 0.015, T = 0), we used mi(0) = 0.1, 0.2, • ■ ■ , 0.9 and r(0) = 1 as 
initial conditions. Then, trajectories finally tended toward the correlated attractor in 
both the numerical integrations and numerical simulations, except for mi(0) = 0.9 in 
the numerical integration for which it seems that the precision of the numerical calcula- 
tion is not good enough. Except for the last case, the results of numerical integrations 
and numerical simulations agree fairly well in both the finite and extensive loading cases. 

Therefore, we demonstrate that the results similar to those in the finite loading 
case have been obtained in the extensive loading case. 

It might be considered that if we could construct a physiological experiment to 
observe the transient behaviour of a firing pattern of a monkey, we would be able 
to judge whether an attractor neural network exists in the brain of the monkey. As 
Miyashita and Chang reported, the neurons in the AVT are highly selective towards a 
few of the 100 colored fractal patterns. This implies that the firing rate of neurons in 
the AVT is very low, while the firing rate is set to 50% in the present model. Thus, in 
the future it will be necessary to develop a model with a low firing rate, [TO] so that we 
can compare results of that model with physiological experimental results. 
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Figure 1. Equilibrium states in finite loading case, a = 0.4 and c = 13. The abscissa 
is T and the ordinate is m jU (/i = 1, • ■ ■ ,7). Solid curve: Hopfield attractor. Dashed 
curves: correlated attractor. Dot-dash-curve: mixed state of thirteen patterns. 
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Figure 2. Results of numerical integrations and numerical simulations for the finite 
loading case, a — 0.4, T = 0.04 and c = 13. Initial condition is m = (m ,0, • • • ,0) 
and mo = 0.1, 0.2, • • • , 1.0. Dashed curves: simulation for N = 60, 000. Solid curves: 
theory, (a) Trajectory in space of (mi, 7712). C: correlated attractor. H: Hopfield 
attractor. (b)Time evolution of mi. 
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Figure 3. Results of numerical integrations (a) and numerical simulations (b) 
for finite loading case, a = 0.4, T = 0.04 and c = 13. Initial condition is 
m = (mo, 0, • • • , 0). (a) Dashed curve: mo = 0.15. Trajectory tends toward correlated 
attractor. Solid curve: m = 0.16. Trajectory tends toward Hopfield attractor. (b) 
N = 60,000. Dashed curve: m = 0.16. Trajectory tends toward correlated attractor. 
Solid curve: m = 0.17. Trajectory tends toward Hopfield attractor. 
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Figure 4. Results of numerical integrations and numerical simulations for finite 
loading case, a = 0.4, T — 0.15 and c = 13. Dashed curves: simulation for TV = 60, 000. 
Solid curves: theory, (a) Trajectory in space of (mi, 7712). C: correlated attractor. H: 
point where Hopfield attractor existed, (b) Time evolution of mi. 
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Figure 5. Equilibrium states for a = 0.35 and T = for case of extensive loading. 
The abscissa is a and the ordinate is m M (/i = l,---,7). Solid curve: Hopfield 
attractor. Dashed curves: correlated attractor. Long dashed curves: mixed state 
of three patterns. Dotted curve: mixed state of thirteen patterns. 
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Figure 6. Results of numerical integrations and numerical simulations for extensive 
loading case, a = 0.35, a = 0.01, T = and c = 13. Dashed curves: simulation for 
N = 60,000. Solid curves: DRT. (a) Trajectory in space of (mi, 7712). C: correlated 
attractor. H: Hopneld attractor. (b)Time evolution of mi. 
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Figure 7. Results of numerical integrations and numerical simulations for the 
extensive loading case, a — 0.35, a — 0.015, T = and c = 13. Dashed curves: 
simulation for N — 60,000. Solid curves: DRT. (a) Trajectory in space of (mi, 7712). 
C: correlated attractor. H: point where Hopfield attractor existed. (b)Time evolution 
of mi. 



